ML crash course - Classification
Machine learning crash course 중 Classification model 챕터.
developers.google.com/machine-learning/crash-course/classification
Introduction
Learning objectives:
- Determine an appropriate threshold for a binary classification model.
- Calculate and choose appropriate metrics to evaluate a binary classification model.
- Interprete ROC and AUC
Prerequisites:
- Introduction to Machine Learning
- ML crash course - Linear regression
- ML crash course - Logistic regression
Classification is the task of predicting which of a set of classes (categories) an example belongs to. You can convert a logistic regression model that predicts a probability into a binary classification model that predicts one of two classes.
Key terms
- Binary classification
- Class
- Classification
- Multi-class classification
- Sigmoid function
Thresholds and the confusion matrix
- While 0.5 might seem like an intuitive threshold, it’s not a good idea if the cost of one type of wrong classification is greater than the other, or if the classes are imbalanced.
Confusion matrix
The probability score is not reality, or ground truth. There are four possible outcomes for each output from a binary classifier.
- True positive (TP): A spam email correctly classified as a spam email. These are the spam messages automatically sent to the spam folder.
- False positive (FP): A not-spam email misclassified as spam. These are the legitimate emails that wind up in the spam folder.
- False negative (FN): A spam email misclassified as not-spam. These are spam emails that aren’t caught by the spam filter and make their way into the inbox.
- True negative (TN): A not-spam email correctly classified as not-spam. These are the legitimate emails that are sent directly to the inbox.
When the total of actual positives is not close to the total of actual negatives, the dataset is imbalanced.
Effect of threshold on true and false positives and negatives
When the classification threshold increases:
- both true and false positives decrease, and
- both true and false negatives increase.
Accuracy, recall, precision, and related metrics
Which evaluation metrics are most meaningful depends on the specific model and the specific task, the cost of different misclassifications, and whether the dataset is balanced or imbalanced.
ROC and AUC
If you want to evaluate a model’s quality across all possible thresholds, you need ROC curve and AUC.
Receiver-operating characteristic curve
Area under the curve
AUC and ROC for choosing model and threshold
Prediction bias
Multi-class classification
Programming exercise
https://developers.google.com/machine-learning/crash-course/classification/programming-exercise